UNFPA

It has been estimated that at a cost of about $1.7 billion a year, modern contraceptive use prevents 105 million induced abortions and another 3.6 million infant and mother deaths (in 2004) per year, with the United Nations Population Fund (UNFPA) being a main provider of contraceptives in developing countries where access would otherwise be limited. If you believe that life begins at conception, than the UNFPA has almost certainly prevented more deaths than any other organization on earth, and with more efficiency (between $7 and $177 per death prevented) than any other organization as well. The tragic irony is that funding cuts to the program in this decade have largely been driven by pro-life organizations, the very ones that hold this view of life (due to misconceptions that the UNFPA advocates abortion, which continues). This isn’t intended to be a slant against the well-meaning intentions of these organizations, but a lesson that to honestly pursue a cause, we must humbly learn from empirical feedback what are effective solutions rather than simply pushing forward with our ideological assumptions.

You can give to UNFPA here.

Capitalism is a Tool

Free market capitalism is awesome. But capitalism is a tool. Arguing about whether it is universally better than some other tool (socialism, communism, restricted capitalism, etc.) is as foolish as arguing about whether a hammer is a universally better than a screwdriver. The more relevant question is when it is appropriate (or in what form). Capitalism works fantastically well with a few conditions that create incentives that drive all parties to make decisions that benefit everyone:
1. Consumers have the ability to make a rational choice about what to consume.
2. Consumers have the freedom to choose between different products or services, creating competition that drives producers towards better value.
3. Demand drives producers to increase supply, benefiting consumers.
4. Consumers inability to afford a product/service does not represent an ethical violation.

When these conditions are present, capitalism generally contributes to a flourishing and just economies. For example with cars, consumers can easily research and test drive car to make a rational choice, they can choose between different makers and models, the demand pushes auto makers towards building more cars and increasing availability, and in general their is no basic inviolable human right to having a vehicle. This is a win-win situation, incentives benefit both producers and consumers in an relatively fair and accurate way that generally generates high and robust growth and production. Capitalism was integral in the industrial revolution were many growing sectors matched these conditions and showcased the benefits of capitalism in spectacular fashion.

However, if any of these conditions are missing, the effects can be negative. When a condition is missing, an otherwise free market may actually create perverse incentives rather than beneficial incentives. For example, when a monopoly occurs, consumers no longer have freedom, thus anti-competition laws are enforced. While many economic sectors fulfill these conditions, it is naive to assume that all do.

Education is sector where there is general consensus that it is ethically unacceptable to deny some children education due to economic circumstances (especially since such circumstances are usually beyond the control of the children). While a free market still exists with private education, allowing consumers to choose potentially superior services, this sector is supplemented with a public social structure because the ethical condition of capitalism is not fully realized. It is not appropriate to apply it as the only tool, thereby denying some children access to education.

Health care is a sector where consumers intrinsically fall short of rational and free choice. Some services are rendered in distress or emergency where there is little opportunity for research or alternatives. Other choices are made by specialists where the buyer isn’t involved (there isn’t a budget/value force towards low-price) or doesn’t understand the options enough to participate. Decisions also involved short-term vs long-term benefits where humans are notoriously poor at make rational decisions (we tend to choose short-term gains with little concern for long-term). Also insurance plays a buffering role further removing the consumer from direct value-based decisions. With these conditions for capitalism missing, perverse incentives are present, motivating increasing health care costs rather than reduction in costs. Predominantly privatized health care has struggled to produce good value (compared to government provided universal health care), not because capitalism doesn’t work, but because the conditions for capitalism simply do not exist (and it’s unlikely that they can be forced to exist) in health care. This is blatantly evident with the US health care system, one of the few developed countries still languishing with a privatized model. The US spends vastly more than any other country on health care, costs are rapidly increasing (since incentives are driving prices up instead of down), to the point where statisticians use it as an example of statistical outlier. In fact the US government spends more on our privatized health care (where the majority of expenditures are private), than do countries with universal health care (where the goverment covers the majority of costs), all while we have shorter life spans and higher mortality rates than most other developed countries. Finally, the health care also fails to meet the ethical condition. Denying basic health services due economic circumstances is a moral failure, even if the system was working efficiently.

Computers and the Internet are also creating situations where the demand-driven supply condition is started to evaporate in certain sectors. An example of this is the music industry. Duplication of music (and other products like software) has effectively reached zero-effort. This means that supply can be almost infinite as soon as music is recorded, as it can be distributed with no virtually effort from the producer. Consequently, perverse incentives are present, leading the music industry to create arbitrary constraints (DRM). With the absence of this condition for capitalism, the music industry is incentivized to create demand by reducing/constraining supply (a negative utility to society), rather than working to increase supply (benefitting society). The solution probably is not socialize the music industry, but we should acknowledge the suboptimal efficiency, and recognize that alternate post-capitalistic models may need to be exercised (and perhaps are already being used, many artists have opted for more of a gift economic approach ).

Of course there will be legitimate disagreements on the degree and type of different economic models needed in different sectors, but we should at a least start with the right question. We must start with a proper perspective of economic models as tools, not sacred institutions, that can be applied in differing degrees in different situations, rather than falling for simplistic overarching false dichotomies. Rather than leaning on our assumptions, the question of what tool to use should be driven by pragmatic look at works for the situation.

Gratefulness and Gas

A gallon of gas contains 130 million Joules of energy, equivalent to the energy of lifting a man from sea level to the top of Mt Everest about 15 times, or 31,000 Calories, and equivalent to about two weeks worth of food. Being able to purchase something with this much energy for less than $30 is incredible and something that we should be extremely grateful for, and we may not always enjoy.

The price of gas is based on simple global supply and demand. Gas comes from oil, which is a fungible resource readily interchangeable on the global market. Supply is dominated by OPEC. The US portion of the supply is very small, it currently produces around 8% of global oil, and holds only 1.5-2% of global reserves (there are some debates on extents of provable reserves, so it is possible we have up to a percentage point higher). On the otherhand, the US provides a significant part of the demand (25% of global demand), thus the US’s primary influence on prices is in demand. Economic recession decreases activity thus decreasing demand and prices, economic growth increases prices. If you don’t like prices, just like any other commodity, the recourse is it to not buy. If you are a buyer you are a participant in demand.

There are a few other contributions to gas prices that either small, regional, short-term. One is the gas tax. George H.W. Bush raised the federal gas tax to 18.4 cents per gallon in 1993, and it has remained at the level ever since. This tax represented about 14% of the price of gas under (the first) Bush. With current gas prices, this federal tax now only constitutes about 5% of the price of gas. The gas tax under the Obama administration is also at the lowest amount in constant dollars in over two decades. This is also one of the smartest taxes that we have since it not only improves roads, but incentivizes lower consumption better than any MPG mandates can (I would love to see it increased). Oil futures and trading affects price, but this too has minimal long-term impacts, and can help cope with supply fluctuations. Like other global products, inflation and exchange rates affect the current price, but the last few years have seen lower than average inflation and a general strengthening of the dollar against other currencies. Finally, there are different state and local taxes, and different regional requirements on the quality gas, and different transportation costs due to proximity to refineries and sources of production, and these are the primary drivers of the differences in gas prices in different locales, but this isn’t related to the overall country gas price trends. For each of these factors the federal government has little control or avoided any price increasing policy changes.

To grumble about prices and make it political is generally either ignorant (the principle way the US reduces prices is through economic recession), or manipulative. Drilling or pipelining more has a negligible affect on global supply. And the US is depleting it’s proven reserves at least 4-5 faster than most countries in the world, drilling faster now as way to avoid foreign dependence on oil just shifts even greatere dependence to our next generation. As one concerned for children’s future, I don’t want to dump them further them into the dead end of oil addiction.

Specifically the supposed benefits of the Keystone pipeline towards lowering prices are particularly vacuous, pipeline don’t produce oil, they just move it around, and as many have pointed out, it will move oil to the gulf for easier export, making it likely to actually slightly increase gas prices in the midwest. The real question in regards to Keystone is simply the ethics of increasing efficiency of tar sand based oil transportation versus the ecological impact.

In reality, increased gas prices are basically due to increased economic activity and recovery (the dominant supply factors are mostly out of our control), increased spring/summer driving, and supply concerns with possible conflict in Iran. The primary effect America has had is in economic recovery (and possibly we’ve played a part in some short-term supply concerns by threatening military action against Iran). You can’t eat your recovery and have your cheap oil too.

When it comes to amazing amount of convenient energy in we can buy in a gallon gas, gratefulness is a better attitude than complaining and reduced usage is a better response than political manipulation.

Kony 2012

A few thoughts from watching Kony 2012: This is an awesome and inspirational video and awesome cause worth fighting for. It is fantastic to see attention brought to tragic foreign issues that affect humanity beyond what just impacts our own self-interests. I know there have been criticisms of Invisible Children, but from what I have seen they have done a great job of responding to critiques and disclosing financial and operational information. They are definitely in it to end this injustice.

However, hopefully we don’t end with Kony. Kony and the LRA are just the tip of the iceberg of global injustices. What Invisible Children has courageously demonstrated is a willingness to learn and expose injustices that have been hidden, and speak out about them. We are not truly following in their footsteps if all we do is share or watch a viral video. And it is shallow pursuit of justice if we only react when there is a clear villian we can point our finger at. One of the criticisms of the film is that it oversimplifies the issue, but in reality this should be a criticism of us, and the fact that we often won’t respond to any issue that is more complicated than what a five year old can digest, even for injustices they have claimed far more lives than Kony. Simplification is just what IC had to do to connect with us (and obviously it worked).

Anyway, let’s follow Invisible Children’s lead. Let’s stop Kony. Our voices do indeed matter, and make a real difference. And let’s not stop there, let’s follow their lead in truly learning about other injustices and making our voices heard. And we can indeed reshape human history, both in Uganda and in the rest of the world.

“Where you live should not determine whether you live… At the end of my life I want to say that the world we left behind is one [my child] can be proud of… A place where children no matter where they live, have a childhood free from fear.” – Jason Russell (cofounder of IC)

A Broken Heart for Valentine’s Day

I wrote a little overview of our Valentine’s day gala we put on for Love146, to fight human trafficking, and submitted it to our local paper (hence the faux journalistic style, and Utah specific comments). They never published it, so here it is:

This last Tuesday, a unique Valentine’s party took place. While couples across America were packing restaurants to celebrate their relationships, about 30 couples gathered in a small church in east Sandy, with a little different focus. They also came together because of love, and also enjoyed a catered dinner and live music, but the gathering was not the typical Valentine’s evening of romance. It was focused on the heart breaking realities of human trafficking. This was an awareness and fund-raising effort for an organization called Love146, to defend, protect, and restore children caught in the tragedy of the sex slavery industry.

The reality of modern day slavery is indeed heart-breaking, even shocking. We learned that approximately 27 million people are enslaved today, generating $32 billion annually (second most lucrative crime activity). The majority of female victims are in the sex industry. An estimated two children are sold every minute. We heard a story of the torturous life of a girl caught in sex slavery, left disfigured from beatings.

This Valentine’s day, attendees also learned how Love146 was started in response to this tragedy. The founders went overseas on an investigative trip to a brothel, to see trafficking first hand. Young girls were contained behind a window, despondently waiting for their next customer, each girl reduced to a number on a menu. However, one girl stood out to them. One girl stared intently back, she still had fight left in her. That girl was #146, and from that encounter the organization was born.

While these stories left many in tears, there was also reason for hope. This fund-raising effort had a substantial impact. In fact the final presentation of the night showed the restoration homes that has been built to protect and nurture children that have been rescued out of slavery.

But surely this doesn’t happen here in America, right? Sadly, about 100,000 US children are forcefully engaged in prostitution or pornography, the average age a female enters prostitution is 12-14. And even here in Utah, there are have been hundreds of cases of trafficking, and some of the highest pornography usage in the US, fueling demand.

Dinner attendees aren’t the only modern abolitionists, you too can be a part of this movement, and make a important impact even with a small amount of time and money. Again, international exchange rates mean donations to groups like Love146 go a long way to tangible prevention and rescue efforts. You can join the local effort in Utah by supporting the local organization Operation61 (they host an annual race in Liberty Park called STOP TRAFFIC). Even without money, you can make a difference. Learn more. Make others aware. Call your US senators Mike Lee and Orrin Hatch, and your representatives to urge them to keep US supply chains slavery-free, use diplomatic efforts to protect victims, and fund the Office to Monitor and Combat Trafficking. This office is a tiny expense (we spend roughly 333 times as much to fight drugs as we do to fight trafficking), and non-partisan issues, so even moderate pressure can have an tremendous impact.

I was proud to be a part of this evening, and I wanted to commend the generous sponsors that made the evening possible, including SitePen, GSBS Architects, PurCo, SDI, USANA, the Point Christian Church, Chick-fil-A and caterers, Good Day Catering. I am also grateful for being part of a faith community, Sandy Ridge Community Church, that supports putting faith into action, pursing the Biblical call to fight injustices.

Sometimes a broken heart on Valentine’s day is a good thing!

Next Mobile Features

I wanted to suggest a few features that I believe could be significant in the mobile (web) space:
* Adding “force” and/or “radius” support to touch events. I believe this was already in the W3C specification at one point. Being able to detect the intensity or force of a touch event opens up entirely new possibilities of touch interaction. Developers could explore many new paradigms for user interaction with this extra information. Let’s not abandon this.
* An web/JavaScript interface for triggering and controlling haptic feedback. Android (and maybe others?) have made some effort to provide haptic feedback in the form of vibrations in response to button/key presses (and have native APIs for this). However, there is a lot of improvements and innovations that could be made if (web) applications could trigger the feedback themselves, providing their own possibilities for feedback based on application specific conditions. I would also love to be able to dictate something more natural and brief than an vibration that lasts for a couple hundred milliseconds. A single oscillation might be more appropriate for button or keyboard press, and longer vibrations might be more appropriate for other actions.
* Touch hover events. Yes, hovering my finger over the touch screen should trigger hover events. I know you are thinking that a touch screen that can’t detect your finger until you touch it. But come on, capacitance can be detected in objects without actual contact by simply moving to a sufficiently high frequency. Inductance at distance is a pretty basic electrical principle. Surely a touch screen could be engineered to detect finger hovering.
* Apps should be able to register as a MIME type handler with the browser so the browser can negotiate MIME types and trigger an application directly in response to links that an application can handle. For example, my Twitter application should register application/vnd.tweet. The browser should then add application/vnd.tweet to the Accept header, and if a server returns an application/vnd.tweet response (as twitter.com should do if application/vnd.tweet is in the Accept header), it should be handled by my Twitter application.
* Detachable/replaceable camera lens (not actually about web technology, just mobile devices). I want to be able to detach my phone’s camera lens and attach a real camera lens. I don’t need an SLR level lens, just one of the low-end lenses like on a typical $100-$200 camera. The 8PM CCD one my phone is more than capable of capturing a good image if it had a decent lens on it, something bigger than you can fit on a phone. I just want to be able to carry an replacement lens if I am know I am going to take picture, and be able to snap it on as needed. It is fine if I have to manually focus it.

Imperative, Functional, and Declarative Programming

November 8th, 2008
Much of the efforts in modern programming language evolution are focused on moving beyond the low level programming paradigm of imperative programming and into the realm of functional and declarative programming. Imperative programming is simplest way to interact with hardware and generally forms the foundation of higher level constructs. However imperative programming is highly error-prone approach to programming and can obstruct optimizations and implementation opportunities that better abstractions provide.

Functional programming focuses on computations and aims to minimize mutating states. Functional programming avoids side-effects and changing data as the basis for operations. Many computations that might rely on changing variables and state information in imperative programming, can be described in terms of mathematical computations in functional programming. Functional programming may rely on imperative steps in the implementations, but these imperative actions are ideally abstracted. For example, lets take a simple loop that finds all the objects in an array with a price property under 10. In imperative programming, this would be:

function bestPrices(inputArray){
var outputArray = [];
for(var i = 0; i < inputArray.length; i++){
if(inputArray[i].price<10){
outputArray.push(inputArray[i]);
}
}
return outputArray;
}
Can you spot the state mutations that we had to induce in this imperative approach? For each loop we had to modify i (increment it) and for each match we had to modify outputArray (push appends a value to it).

Now if we use a more functional approach:

function bestPrices(inputArray){
return Array.filter(inputArray, function(value){
return value.price < 10;
}
}
This code is completely free of side-effects. We simply described the operation in the form of expressions. The underlying implementation hides all the imperative actions necessary to make this work.

Declarative programming is defined as programming in terms of "what" instead of "how". Functional programming can be categorized as a form of declarative programming since we describe "what" computation should take place with expressions, instead of describing how the computation should be performed in a series of steps as in imperative programming. However, the declarative programming concept goes beyond just functional programming.

One of the central aspects of most applications is persisting data. A core service that most applications provide to users is to be able to store information relating to the use of the application. A social networking application is useless if it can’t actually remember anything about you and your friends. The functional paradigm is limited here, it’s methodology stands in complete opposition to the central goal of the application. Quite simply, applications must deal with mutating states and data. Here we can still apply declarative programming concepts, describing what the data is, instead of how it was created or how we modify it. Perhaps the most critical and invaluable benefit of declarative programming is that the representation of "what" the object implicitly dictates"how" the object is modified . With a declarative approach there is no need for describing how we interact with and modify data, because the structure of the data reveals the how without any further instruction.

JSON provides a great example of declarative programming. JavaScript supports numerous imperative techniques, but JSON is a declarative subset, which precisely defines what an object’s state should be without directions on how to get it there. For example, we could use imperative techniques:

var movie = new Object();
movie.title="Transsiberan";
movie["rat" + "ing"] = 1+2;
movie.setHowLong = function(len){
this.length = len.trim();
}
movie.setHowLong("2 hours ");
This representation describes how to build the state of the object, and can be of unbounded complexity, but it is not actually a representation of the final state. Alternately with JSON:

{"title":Transsineran",
"rating":3,
"length:"2 hours"}
The application of the concept of representation implicitly defining the method of modification can be seen here in the simple JavaScript object. The object’s state is observable as a set of properties and we can immediately infer from the presence of these properties how we modify the objects. We can simply set one of the existing properties to modify the state, and the effect of setting a property is obvious. Conversely, when an imperative approach where opaque object methods are used to modify an object’s state, the effects are not inferrable, one has to refer to out-of-band documentation to understand the interaction necessary to cause a state change.

The other central concept of a declarative approach to data is reversibility . If we modify the state of this movie object, there is a clear unambiguous easily computable representation for the new state of the object with a declarative definition. However, this is not true with the imperative definition. With the unbounded complexity possibilities of the imperative representation, it is impossible to always know exactly when step in the directions needs to modified to correspond the state change.

Another example of declarative programming is HTML. HTML represents the state of the layout of the page. The steps for how to get into a particular layout are not described, the layout is a state that is represented by the HTML. HTML provides clear reversibility as well. Layout changes can be clearly and directly mapped to changes in the HTML representation.

Applications themselves are a form of state as well, both at the low level (code is occupies memory in the form of binary data) and at the high level in languages that reify code. JavaScript is an example of an language where code is data. Here as well, we can benefit from code being organized in declarative approach rather than an imperative approach. An application runtime state that is the result of confusing imperative steps is much more difficult to debug than application that has a clear declarative organization.

Furthermore, declarative programming can often allow programmers to work within a "live" interactive environment, where code can be changed on the fly without requiring a restart. Imperative programming often means that the only way to predictably see the effect of a change in imperative code is to restart. We certainly want to be able to move towards faster development techniques that aren’t stuck in windows-style "restart" cycles.

Once again, declarative programming can be built on imperative programming. If strict discipline of rules is applied to imperative steps, the consistency can yield an declarative form in the context of rules. For example, if we said that all class declarations must take the form assigning a constructor function to a variable and then assigning an JSON object to the constructor’s prototype, then within the context of these rules, we can have a reversible representation that still be used even if we change the state of the class (changing method implementations for example).

It is important that these principles be used to direct our continuing evolution of JavaScript. Techniques like pseudo-classes based on the side-effects within constructor function execution are a regression in the evolution away from imperative to declarative. Prototype-based inheritance does a great job of maintaining declarative programming, as the application itself can be continually updated in interactive code. Debugging can be "live", classes can be evolved without requiring environmental restarts.

JavaScript Expressions – Beyond JSON

September 22nd, 2008
JSON a powerful simple expressive format for data structures with a high level of interoperability; implementations exist in virtually every popular language. However, there are certainly situations where developers often want additional constructs for effectively representing data that are not natively supported in JSON. Perhaps the most common usage of JSON is for the consumption of data in the browser. Most JavaScript libraries, parse JSON by using eval, and consequently are actually capable of full JavaScript expression evaluation, of which JSON is a subset. JavaScript expressions support a much wider range of constructs than pure JSON. Usually a simple JSON/JavaScript expression parser looks like:

function parseJson(jsonText){
return eval(‘(‘ + jsonText + ‘)’);
}
One of the most oft-desired data type that JSON doesn’t provide is a date type. Numerous creative, bizarre, weird, and silly techniques have been proposed for expressing dates in JavaScript. These methods often require extra parsing or walking strategies. Douglas Crockford’s reference library for JavaScript JSON serialization, serializes dates to strings by ISO format. I have written about deserializing these ISO dates. But, on deserialization it is not possible to determine if a value is actually a string or is really intended to be an actual date object. However, if the recipient of JSON is known to support full JavaScript evaluation (like the browser with a library using an eval), the solution for delivering date type value is simple, we can just use the normal JavaScript constructor:

{“mydate”:new Date(1222057313264)}
There are also several numeric entities that JavaScript provides that are not available, including Infinite, -Infinite, and NaN. I am not sure why you would need it, but undefined can also be transferred with JavaScript. Functions can be included as well:

{
“reallyBig”: Infinite,
“parsedString”: NaN,
“whatValue?”: undefined,
“doSomething”:function(arg){return doSomethingElse{arg};}
}
The data representations thus demonstrated are not JSON, they are JavaScript using the same object/array literal that JSON is based on. However, certainly one of the biggest benefits of JSON is it’s interoperability. If your data is going to be consumed by more than just JavaScript-parsing JavaScript libraries, you must make your data available in pure JSON format as well. This is well-handled through content negotiation. If you are using JavaScript expressions to transfer data, you should make sure your requests from the browser actually are specifying they can handle JavaScript:

Accept: text/javascript
Your server should be prepared to handle requests from clients that indicate that they only understand pure JSON:

Accept: application/json
Persevere, the JavaScript/JSON application and storage server, also provides support for parsing and storing extra constructs including non-finite numbers (NaN, Infinite), functions. Persevere can output the data as JSON or JavaScript (expression).

JSON is certainly powerful, expressive data format. This post is by no means an attempt to expand or lobby for the modification JSON. However, when it is known that consumers are actually JavaScript capable clients, it can often be advantageous to use the full power of JavaScript to represent data, while still providing JSON as representation for non-JavaScript capable clients.

The Next Great Protocol: HTTP

February 12th, 2008
I suppose this post would be more prophetic a decade or two ago. It was in the 90’s that the HTTP protocol really became the Great protocol. It is foundation of the World Wide Web, and is language on which browsers were able to really open the doorway to the Internet for us. So am I little behind the times to suggest that HTTP now has an emerging future relevance? Is HTTP a relic of the past or does it have something to contribute to the future?

One of the distinctives of current Internet technological advance is in the growing realm of open sharing and utilizing data from disparate sources. Facilitating this progress is one of the principle goals of my work and this site. I want the Open Web to be more than just a bunch of pages that are developed without proprietary constraints, but for the Open Web to be the environment for open flow of information with intelligible interconnection of data that can give participants unprecedented leverage and permutations of capabilities. Mashups are a buzzword to describe this process. However, in order for information to flow rapidly, there must be commonly understood communication. JSON has enormous potential because it is so simple, expressive, and pervasive that it forms a excellent syntax for expressing data. However, JSON is not a transport. Two agents that wish to dialogue may understand JSON data, but they still need a mechanism to communicate and transfer that information. HTTP is almost ubiquitously the right choice for the transport. The incredible adoption of HTTP is main reason for this. No transport is more widely understood

What is wrong with HTTP?

Before going into the benefits of HTTP, let us look at the problems with the HTTP, or more specifically, THE problem with HTTP. The most fundamental problem with HTTP is that it requires that every single response must be preceding by one corresponding request. The specification describes HTTP as request/response protocol. This constraint has an enormous impact on the capabilities of HTTP. The first major impact is in hindering performance optimizations. In order to load a web page, every resource must be requested before the server can send the resources. This creates a signficant latency problems. There can be large gaps in transfers while servers are waiting to receive requests. However, a server could easily determine the most likely resources that a user agent will need and send them before the request if this constraint was not in place.

This constraint is also the fundamental cause of consternation with Comet development. Comet consists of efforts to allow servers to send messages to clients asynchronously instead of in immediate response to a request. Doing this requires creating an outstanding HTTP request that the server can respond to when it wants to. Comet push capabilities could easily be achieved if servers could simply send messages to the client without requiring a preceding request.

Throwing away the entire protocol because of a single issue is absurd, rather let’s fix or enhance the protocol. In recent articles I have discussed how non-request/response-bound HTTP messages can be sent within HTTP messages in order to deal with this problem using existing infrastructure.

So why is HTTP the right choice for future information transport?

It is so pervasive – HTTP is everywhere. It is how all browsers communicate with web servers. HTTP is understood by an overwhelming amount of software. Attempts to reinvent the functionality and capabilities of HTTP is essentially asking for this broadly understood language to be ignored in lieu of new one. Without very significant advantageous to new semantics and vocabulary, such attempts are generally either doomed to obscurity or worse, a cause in division in multiple semantics for the same thing, causing increased code complexity and costs.
It has the constructs for tomorrow – With ever increasing interchange of data in the future, more sophisticated and robust techniques for communicating data are needed. Many of these techniques already exist in HTTP, but have simply not yet been needed with yesterday’s technology. The future of high performance, intelligent data interchange will hinge on capabilities that already exist in HTTP including:
Content negotiation – HTTP includes vocabulary for negotiating between different formats.
Partial data transfers – HTTP has an extensible mechanism for sending a range of information.
Robust error handling – HTTP includes a comprehensive set of errors.
Parallel scalability – Perhaps one of the most impressive features of HTTP is how carefully designed such that demand can be easily scaled across numerous machines as HTTP proxy servers.
REST/CRUD semantics – HTTP provides semantics for basic create, read, update, and delete operations.
Performance improvements – HTTP pipelining has only begun to be utilized (only Opera has it turned on by default). Substantial performance improvements can be realized through pipelining.
Emerging technologies give us new leverage with HTTP – With traditional web application development, much of the workings of HTTP were hidden away by the browser and the server. However, the Ajax revolution gives developers far greater control of HTTP messaging. Most Ajax developers have simply used XMLHttpRequest as a means for communicating simple payloads of data back and forth to server, but XHR has given developers new access to the HTTP capabilities through their header metadata, and leverage to utilize the full semantics of HTTP for more meaningful communication.
Furthermore, with new XHR capabilities coming soon (FF3 will have cross site XHR support), XHR communication will involve much more than simply communicating with your own server. Communicating with your own server does not necessarily require widely understood vocabulary, but when communicating with other servers, the cost and efficiency of integration will be directly related to how much shared vocabulary can be utilized to provide jointly understood communication. New ways to utilize, extend, and leverage HTTP are being developed as well like the Atom Publishing Protocol.
Does it really matter what is on the wire? Can’t we simply distribute API based communication handlers? Consider this, is it easier to setup a TV to tune into the available TV stations, or is it easier to setup a printer to work with your new operating system? TV stations have standardized on a single format for broadcasting content. Connecting a TV to a station is as simple as turning it on choosing your station. On other hand, printers have no standardized communication with servers. Devices like printers use API based communication handlers (AKA drivers). With a huge number of different protocols for each printer, and different operating systems to interact with, there are an enormous amount of permutations of different drivers that must be developed, very prone to incompatibilities. While the situation has improved, many have experienced the frustrating effort that can go into trying to find the right driver for your OS/printer combination.

But in the realm of browser communication does it matter since we are all using JavaScript? Absolutely. There may be a single language on the browser, but there are different JavaScript libraries, with may prefer different APIs. In addition the browser environment can be very bandwidth constrained. Requiring another library for each data endpoint that you want to connect to, does not scale well. And as we move towards more service oriented architectures, browsers will not be the only consumers of information. Many other clients must be considered as well.

We need to be examining the HTTP specifications and learning how to leverage the power that is available, to maximize our potential in coming world of extensive data interchange and mashups. Next time you need to create Ajax communication to trigger CRUD operations, consider using the RESTful HTTP methods (PUT, POST, and DELETE). Do you need to deliver multiple formats of the resource? Consider using HTTP content negotiation. Do you want Comet capabilities in your application? Consider using an HTTP standards based approach. The more we can utilize what is there, the more widely we can be understood, and the more efficiently we can utilize the infrastructure of the web. The full utilization of HTTP can provide a solid foundation for the future of data interchange.

JSONP Header Transfer Proposal

December 24th, 2007
JSONP is a proposal for performing cross site JSON data interchange by using script tag insertion with a callback to deliver the data payload. JSONP can be used in place of XmlHttpRequest in cross site situations. However, when using XmlHttpRequest, there are situations where the JSON data in content body is not the only relevant information, but the response headers, as well other aspects of the HTTP request and response may also contain important information. I propose that JSONP can extended in a small and simple manner to accommodate the transfer of header information in suitably interoperable manner.

I propose when custom headers need to be included in the request, that headers simply be included as parameters. When servers that are responding to JSONP requests want to include response header information in the response, they may optionally respond with a second argument that contains header information in the form of an JSON object/map. The object should contain properties with the names corresponding to header names, and values corresponding to header values. The request format can remain the same:

http://myserver/requestJSON?callback=mycallback&Accept=application%2Fjson

And the response would look like:

mycallback({“name”:”foo”},{“Content-Type”:”application/json”,”Expires”:”Mon, 24 Dec 2007 17:09:04 GMT”})
This addition to JSONP should generally not break existing JSONP clients, as normally only one parameters is used. This allows JSONP to emulate more semantically correct manners of transferring information that conceptually belongs in headers.

Usages could include transferring the “Last-Modified” header, so clients could determine the freshness of the data, sending If-Modified-Since header in requests and many more.

In addition to transferring request and response headers, I propose that additional HTTP information could also be transferred in pseudo headers. The following pseudo headers may be used:

request
httpMethod – Indicates the HTTP method used (GET, POST, PUT, DELETE, etc).
httpContent – Indicates the body of the content of the HTTP request (such as the POST body)
httpNoCache – With non-idempotent methods, you should generally include an additional parameter with a random unique value to ensure that the request is not cached.
response
httpStatusCode – Indicates the HTTP status code.
httpStatusText – Indicates the HTTP status text.
A more sophisticated example request could be:

http://myserver/requestJSON?httpMethod=POST&httpNoCache=23n9zs92l&httpContent={“name”:”bar”}&callback=call1
And the response could be:

call1([{"name":"foo"},{"name":"bar"}],{”httpStatusCode”:200,”httpStatusText”:”OK”,”Date”:”Mon, 24 Dec 2007 17:09:04 GMT”})
Of course, this enables a greater level of interaction, and therefore the prerequisite security warnings must be heeded. You can get yourself in a lot of trouble with XSS if you are not using proper explicit authentication schemes.

Follow

Get every new post delivered to your Inbox.