Nostradoofus

The U.S. Constitution explicitly forbids cruel and unusual punishment. What is cruel? One way to nail it down is to compare alternatives. Here is an instructive thought experiment.

Imagine our judicial system was required to provide nonviolent convicts a choice between two types of sentence. One would be a normal prison term. The other would be an “equivalent” amount of caning (a form of extremely painful whipping currently employed in Singapore, which in turn adopted it from the old British colonial justice system).

For example, say you are convicted of felony drug possession, and sentenced to 6 months in jail. Under this imaginary system, the judge would be required to offer you the option (but not the requirement) to receive, let’s say, 20 lashes instead of prison time.

For those who don’t know, caning is horrifically painful. Brutal. Barbaric.

And yet, which would you choose? Obviously the caning. Why? Less dangerous than jail. No beatings, rape, race riots, corruption, extortion, or threats. Millions of Americans face these things — things nearly anyone would do nearly anything to avoid — in prison every day.

Thus, while we can’t say the prison system is cruel in an absolute sense, we can say that any reasonable person considers it more cruel than a brutal whipping. To me, that sounds pretty cruel.

Now for the irony:

1. Behavioral psychologists will tell you caning deters crime better than prison.
2. Judges will tell you the appeals court logjam would vanish if convicts had an option to accept corporal punishment. The entire justice system would be, faster, fairer and more effective.
3. The prison population would fall by more than half, releasing billions of dollars into the productive economy, helping to balance government budgets, etc.

In short, corporal punishment by choice (not by obligation) is cheaper, more effective, and preferable to both the justice system and to the convicted. The very idea is repugnant, and yet any of us would choose it over our existing prison system. Very strange situation.

—

I generally don’t sell stocks short, due to unlimited downside and exposure to speculative bubbles. You can be right in the long run, yet get killed by a margin call in the short run.

That said, if I were to sell something short now, it would be the entire state of Florida.

Regional real estate wipeouts all tend to repeat a certain trajectory:

1. Buyers vanish, but sellers are unwilling to drop prices.
2. Consequently, the number of sales collapses.
3. A year or two goes by.
4. Sellers start giving up and reduce asking prices.
5. Buyers see prices falling, and decide to wait it out.
6. Sellers are forced to reduce prices further.
7. If seller’s equity falls below zero, they stop paying mortgages.
8. Bad loans proliferate.
9. Banks with geographically concentrated loan portfolios start going broke.

This has happened several times in regional markets in the U.S. — Houston is the most flamboyant example of the last 30 years.

In Florida, we have arrived at Step 2 (see above) with the recent report that the number of sales in Broward County in south Florida has fallen 34% from last year. Median price fell less than 1% — we’re now waiting to complete Step 3.

This problem may extend to other states as well. But Florida looks particularly bad, because:

• The state’s economy is unusually dependent upon real estate.
• There exist many vacation communities with little or no underlying economy.
• Rampant speculation, including the infamous CondoFlip.com.
• Prices rise at 30% per annum from 2000 to 2005.
• Insurers flee the state on fears giant hurricanes will be common.
• Even Bush’s EPA puts south Florida under water in 100 years.
• South Florida will suddenly run out of water within 20 years, as seawater invades the aquifer supplying most of its drinking water.

Now, it’s possible Miami will become a charming, windblown Venice, connected to the mainland by a 100-mile, hurricane-proof causeway, giant desalination plants humming 24/7. Then again, I wouldn’t bet my own money on that.

—

Summary

There are two approaches to promoting websites, one adversarial and one
cooperative. You can either fool search engines into thinking you are
useful, or you can actually become useful, attracting search engines and
users. Both can work. But in the long run, the latter is more effective
with less effort.

My qualifications

I’m not a recognized SEO expert. I don’t consult or sell books. This is
the only thing I’ve written on the subject. Yet in a few years, in my spare
time, with a budget of zero, I have sent several websites to #1 at major
search engines, generating 50,000 to 100,000 page views a month.

Since it cost me nothing to learn this, I will now tell you, for free, how to
do it.

Do this for each site

1. Choose a domain name that describes the content.
2. Put interesting and useful content there.
3. Get listed at www.dmoz.org.
4. Describe each page clearly and succinctly in the page title and body.
5. Get links from established, reputable sites with similar content.
6. Wait several months.

That’s it. You don’t need to hire anyone, nor buy any tools or books.  This method works for nearly every site, nearly every time, for years, with almost no maintenance.

Why it works better

This approach arises from a simple guiding principle:

Understand how search engines work, and what they want to do — and cooperate.

Traditional promoters want to use Google like a megaphone, because that’s how news media works these days. But Google is not a megaphone. It is a listening device.

To really understand that, and to appreciate the cooperative approach to SEO, you need to be in right frame of mind, which by itself can be difficult for the typical person reading an article like this. You very likely work in a frantically competitive job, involving long hours and the threat of layoff or reassignment. This tends to create a short-term, shortcut, screw-you mindset. So first, a brief exercise to distance us from that world.

Preparatory Exercise

As you’re sweating alone over a particularly tedious project at 11pm on a Saturday night, please stand up, look into your bloodshot eyes in the mirror, and read the following:

“Out, out, brief candle!
Life’s but a walking shadow, a poor player
That struts and frets his hour upon the stage
And then is heard no more: it is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.”

If, at this point, you still feel highly motivated to tap away like a chimpanzee, then there’s nothing I can do for you. Try clicking some helpful consultant’s ads — they’ll probably sell you cool, colorful charts or something.

On the other hand, if you are now in tears, bemoaning your wasted youth, you are mentally prepared for cooperative SEO.

Choose efforts that will last

It’s not widely recognized that Macbeth was discussing search optimization philosophy. Do things that will still have value in a year or two, and you’ll feel less like a chimpanzee. Even a tiny effect is valuable, if it lasts for years with no further effort.

The goal in search optimization — the true goal — is to gain a position whose value (measured in dollars, readers, or whatever else is important to you) greatly exceeds the effort required to get there and remain there.

This means working with those features of search engines that are utterly stable, and likely to change slowly or never.

Stable features of search engines

Search engines use a huge and ever-increasing array of techniques to measure importance and relevance, and to filter out scams and spam. But the following have been stable for about 10 years.

1. Your page’s importance is based on the number and importance of links coming in to your page from other pages. 
2. Importance is calculated iteratively, and must begin from a set of initial conditions provided by a reference directory. For Google, the initial conditions are provided
   primarily by www.dmoz.org. 
3. Relevance is inferred from the similarity between a user query and the contents of a page and pages linking to it.

The reason this hasn’t changed in 10 years is that it works. It is a powerful set of insights into delivering what people need. So powerful, in fact, that it’s difficult to improve upon. And difficult to change.

Stated another way, the search engines have solved the search problem so well that they’re now beneficially unable to change their basic ranking approach. That inflexibility means that if you can gain high rank according to the above three criteria, you will tend to sustain that rank indefinitely, with no further effort. That’s the promise offered by cooperative search optimization. Now, let’s look at the alternative.

Gaming the system — beyond ethics, it’s a waste of time

Looking at the stable features above, you can see the potential adversarial strategies: link farms, link exchanges and so forth. Some of these probably do work — for a while.

Let’s say you had no ethical qualms about gaming the system. You’re only concerned about return on investment. Is this an effective way to spend your time or money? Probably not, because you are exploiting an unstable set of loopholes.

The search companies have huge teams of smart guys dedicated to prevent cheating. As a result, adversarial strategies are a constant arms race. You would need constantly to invent new strategies as they discover and shut down your old ones. All your old work becomes worthless within a few months. A sound
and fury, signifying nothing.

Spending your time becoming interesting

By contrast, providing end users with something uniquely useful is a great investment of time or money. To see why, look at the two main hurdles you need to jump in gaining a good search rank: DMOZ and inbound links.

People know DMOZ is critical to a good search ranking, so this creaky all-volunteer Web directory is utterly overwhelmed by requests from websites to get listed. Yes, there are probably ways to sneak in, but the most straightforward solution is to offer something unique, useful and authoritative. The DMOZ reviewer will immediately recognize this in your site, if you have chosen the right category and are truly providing unique utility.

Inbound links are easier to obtain than a DMOZ listing. But authoritative, closely related inbound links are not. You need to create something sufficiently distinguished to warrant attention from people with no vested interest in your success.

Your content is not automatically unique or useful

Note the previous section is not saying, “Spend all your time creating content.” If that were all you needed, every high school student’s blog would get more traffic than the Toyota home page. No, the previous section says to spend effort creating not just content, but useful content. And not just useful content, but uniquely useful content.

My blog (I do write one) is unique, but not useful. As a result, no one links to it, and so it’s mostly invisible on the Web.  Similarly, if you create a website full of recipes, it is likely useful, but absolutely not unique, and thus unlikely to gain a good rank.

You need to think about what people need, what you can provide, how you are unique, and how that uniqueness might be valuable. This is a completely different type of challenge from frantic chimpanzee tapping. It involves talking to people, asking questions, and just sitting in a quiet room, thinking.

Knowing the difference between adversarial and cooperative

Now you know the answer to success on the Web. In the abstract. But in practice, for someone not accustomed to thinking in this way, it may not be obvious which activities are adversarial and which are cooperative. Here are a few examples.

Adversarial (basic intent is to fool a search engine)

Link exchanges offered by mass email.

Buy a domain with existing high rank, intending to change the content.

Cooperative (basic intent is to deliver value to site visitor)

Buy a domain with existing high rank, slowly adding similar content.

Get a legitimate online newspaper to link to your site.

How would you rather spend your time?

In conclusion, your mom was right that cheaters never prosper. But beyond that, why would you want to live like that? Always looking over your shoulder, trying to stay a step ahead of the anti-spam cops. Anyone smart enough to do that successfully for a while is more than smart enough to generate actual content, of actual unique value, to benefit a lot of people.  

Why would you want to do it any differently?

—

Note: If you find this article useful, please link to it, thereby helping to prove its thesis.

—

No comment on the President Bush “plan” of January 31, 2006, except to say that if you have any empathy at all, it’s hard to watch a man make a fool of himself on national TV.

The cheapest solution to the oil quandary requires understanding the problem very precisely. The problem is not oil supply — it is oil supply uncertainty. We do crazy things in the Mideast (e.g. cozying up to dictators, invading Iraq) because we fear the economic effect of unexpected disruptions in supply.

The solution is to build a gigantic strategic reserve, enough to cover all imports under normal demand conditions for 9 months. This would cost many billions, but is less expensive than the alternatives (war and economic shocks).

We need not be able to last forever without oil imports. We need only be able to outlast the exporters. Remember, they are much poorer than we are, and oil is essentially their sole source of income. None can survive six months without selling oil, and they cannot boycott just one country: since oil is a commodity, if they sell to ANY market economy, supply increases and price falls for everyone.

Imagine what would happen if Iran shut down exports under these conditions. The U.S. would experience minor hardship, but after six months, Iran’s economy would collapse. They only reason they wield power now is because our reserve is too small.

A giant reserve provides an economist’s form of independence: we still use oil, we still import, but we are no longer subject to pricing power by the seller. Best of all, this is something that requires no new technology. We could do it right now.

—

A big advantage of open source software is to align the interests of developer and end user.

To see why that’s true, notice how it is not true in commercial software. The competitive dynamics of commercial software push developers to make decisions that are “piecewise competitive” with rivals, even if such decisions create inflexibility, incompatibility or increased maintenance costs in the long run.

For example, and most commonly, a commercial developer may optimize for performance by writing code in ways (e.g. choice of language or data type) that cause long-term inflexibility with respect to future hardware platforms or customer needs. This is in the developer’s best interest, because it’s the only way his firm can survive short-term competitive pressures. However, it is against the user’s interest, because it forces upgrades or incompatibility in the future.

Another example is that a developer may add features that are useful to only a small fraction of customers; this creates a temporary competitive advantage for the developer, because it adds a bullet point to the box in the store, influencing a customer’s purchase. It also raises the barriers to entry in that software segment, which benefits all leading developers. However, this decision often works against the average user’s best long-term interest, because it makes software unnecessarily complex for the vast majority of users, and raises the long-term cost of the software, as barriers to entry rise and competitive pressures fall in the segment.

By contrast, many GPL projects make design and implementation decisions that seek to maximize coder productivity and device independence, at the expense of peformance and features. This is in the project developer’s interest, because it is the only way such a project can exist: without high productivity, maintainability and succinctness, even a successful project would eventually drown in its own bug reports. The coding team is too small to support such problems, and generally cannot expand over time.

This is generally in the end user’s best interest, because it allows simpler, more invisible upgrades to future hardware platforms; reduces confusion over features; increases modularity and stability of behavior over time.

You can extend the argument to cover other noncommercial projects like the original Unix, i.e. it survived into modern times because of some developer-centric design choices (performance-inefficient in the short run but maintainable in the long run) that happened to be compatible with the end-user’s best interest.

Thanks to Jeff Cohen of Genezzo for sparking this thought with the following email:

—

From: Jeff Cohen
Date: Nov 29, 2005 10:47 PM
Subject: Re: Genezzo version 0.53 is now available

If you asked them, they’d probably say it’s impractical, and nobody could design such a schema. Of course, given the precedent of computer-generated queries of insane complexity past human comprehension, I can easily envision computer-generated schemas. Generally, the issue is that languages that C/C++ tend to encourage certain types of design rigidity and premature optimization. For example, to encode the number of tables in a join, they could use an int, or maybe an unsigned int, or even a byte if they want to shave some space. And they might pre-allocate some fixed data structures inside the join row source for efficiency. Oracle was literally riddled with these little”optimizations”, and it caused all kinds of headaches when we needed to expand an existing data structure. I believe I told you about the COUNT(*) overflow at WalMart, where ancient coders couldn’t conceive of a table with over 2^32 rows. With languages like Perl, Ruby, or Python, the native data structures are dynamic, flexible and extensible, so coders are less inclined to construct fixed-size structures for practical and philosophical reasons. And the native numeric type is a generic number –the languages automatically switch representations internally for proper efficiency and precision. For example, Python might start using an int internally, then switch to float, and finally to bignum for arbitrary precision. And finally, since I’ve been bitten by these issues so many times, I am very vigilant in order to avoid getting trapped. For example, even the basic block encoding on disk uses variable-length BER encoding for lengths, so I can use a single-byte length for strings under 128 bytes, and up to 128 bytes of length forBLOBs over 2^1022 bytes. BER is more expensive than a fixed-length encoding, but in the future, if someone wants to shove a 3-D, holographic movie into a column inGenezzo, we should be able to handle it.

Jeff

—

VC investment in GPL projects. The very idea recalls an old military saying about the F-14 Tomcat (or, some claim, the F-4 Phantom): “America’s proof that given a big enough engine, even a brick can fly.”

Certain GPL investments similarly remain commercially airborne with immense investor thrust behind them. But this will be unusual, and the main result will simply be to consume a lot of fuel. Software has been so successful for VCs for so long that it’s taking an inordinately long time for them to recognize the sea change: GPL closes down the old opportunities in software, and creates new opportunities that are not in software, at least not directly. Why? Basic microeconomics, and basic Michael Porter.

Software investments traditionally generate returns in two possible ways. First, you can create high customer switching costs through software complexity and/or ownership of proprietary interchange formats. Second, if you can capture enough of the market to permit enormous fixed investment in your source base, outspending any other potential entrant — economies of scale.

GPL does away with both of these possibilities. Economies of scale become impossible, because your fixed investment is automatically shared for free, no matter how large. Switching costs become nearly zero, because file formats and feature sets are no longer proprietary. In both cases, no matter how big you get, any 15-year-old in his bedroom can download your source code and compete with you.

This does not eliminate market power, it just moves it around. Anybody in the business of selling licensed software (Microsoft, Oracle, etc.) is in decline. They can’t possibly compete with a price of zero — all they can do is slow down the transition through FUD and customer relationships. Over time, all such product business will turn into service businesses.

But the biggest companies can mine their patents. A likely endpoint for Microsoft, 15 years from now, is as a seller of services and a licensor of its vast patent library. How large that business may be is not knowable.

And users of open source will thrive, when they can set up switching costs that are not related to the software itself. Google, eBay and Amazon all have this quality.

Nothing in this article should be construed as yet another dull argument about who is evil and who isn’t. I happen to think Google may become a larger, more powerful monopoly than Microsoft at its peak. That is not to say Google or Microsoft are good or bad — I’ll leave the moralization to the guys at Slashdot, because they’re so good at it.

—

I’m a lifelong, 5th-generation, pro-business Republican. But I’m the first to admit the party has lost its mind.

To see how far the GOP has strayed from its principles, imagine you live in 1965, under the Johnson administration, and are suddenly catapulted forward 40 years.

You arrive in 2005 to find that one party dominates politics. It engages in massive deficit spending. It attempts a guns-and-butter strategy of running a major overseas war while expanding social programs at home. It creates regulatory barriers to protect domestic industries from foreign competition. It expands regulation of financial markets, reducing the efficiency of the engines of American capitalism. It expands federal control over traditional states-rights issues. And its leadership values social engineering over economic advancement.

Now someone asks you: based on that information, which party is in power? You, freshly arrived from 1965, would naturally say Democrats. And you would be wrong.

Republicans are now demonstrably a bunch of regulation-happy, big-government spendthrifts. Where are the laissez-faire, get-out-of-my-face Republicans I thought I was voting for?

Who are these guys?

—

If you reduce the supply of something, the price goes up. The more successfully the US destroys coca crops, the higher the price goes, and the greater the incentive for tropical farmers everywhere to simply grow more.

If you instead reduce demand, the price goes down. The more successfully the US reduces coca demand (whether by moral suasion, education, re-education, imprisonment, testing, etc), the lower the price goes, and the less the incentive to grow, refine and distribute it.

If you focus on demand reduction, you also:

1. Starve bad guys of profits.
2. Automatically reduce a variety of related domestic social problems.
3. Reduce the incentive for corruption of US officials.
4. Stabilize Mexico’s border regions, reducing violent crime in the US.
5. Become a model of self-restraint, worthy of self-respect and the respect of other nations.
6. Save money. Projecting military power is immensely expensive — much more than fixing domestic problems.

Supply reduction accomplishes none of these. Microeconomically, the 20-year “war on drugs” is not only a failure — it is a surprisingly dumb idea.

—

The megastorms of recent weeks revealed that certain industrial resources of the U.S. are highly concentrated geographically. To choose just one example from dozens, the New Orleans flood inundated 25% of the world’s stockpile of zinc, sending commodities markets reeling (source: Financial Times).

More generally, it appears accurate to say that the forms of economic advancement of the past several decades bring lower resistance to infrastructure shocks. Intricate supply networks make us more dependent upon transport, communications and energy delivery systems. At the same time, scale economies and the benefits of industrial clustering both drive geographic concentration.

The result is that our economic status depends upon the joint probability that none of these myriad components will ever fail. That sounds pretty darn risky, over time.

The risk is much greater if we get into fights unnecessarily. Hawks take pride in past US military glory, but may forget it was facilitated by industrial invulnerability. Conditions are different today, and not just because the enemy can reach us more easily. The simple fact is that we can’t take a punch as well today as we could in 1950.

If the port of Long Beach were destroyed in 1950, would the toy industry have collapsed? No. Today? Yes. 85% of toys come from China.

If Manhattan were destroyed in 1950, would the national home mortgage market have collapsed? No. Today? Yes. The entire mortgage business is built around securitization.

If the border with Mexico were closed in 1950, would the U.S. auto industry have collapsed? No. Today? Yes. Key components are built over the border.

The list of such dependencies is almost endless. We are exposed everywhere. As a result, globalization is simply inconsistent with the projection of military power against a determined opponent with little to lose.

Ironically, it’s the isolated economic also-rans that are least exposed, and hence the most dangerous to tangle with. Arab nations. Iran. North Korea. Cuba. Integrating these places into the global trading system would expose them to the same global dependencies that we face — and render it unprofitable for them to threaten anyone.

China, our supposed military rival, is perhaps the most exposed of any large nation. They would be crippled by global industrial network disruption. Thus it’s perhaps not sensible to worry about them from a military perspective — they simply have too much to lose.

Japan’s government may understand this better than ours or China’s. As the first practitioner of many of these modern industrial methods, they have decades more experience at such dependence, and presumably more willingness to prepare for shocks. Is their relative neutrality in world affairs one such preparation? Do they recognize, better than we do, that their way of life depends upon the function of those intricate networks?

—

eBay is not the only deflationary force on the Web. The improved availability of replacement parts has a similar effect.

My toddler son (we’ll call him Baby Godzilla, though that’s not his real name) recently tried to eat our DVD remote. Though he couldn’t bite through the plastic, B.G.’s caustic saliva immediately corroded and disabled the remote. Like most DVD players, ours can’t be used without the remote.

As late as 2002, it would have been prohibitively time-consuming to locate a replacement part for a three-year-old Toshiba product. Net of the value of our time, it was simply cheaper to junk the working player and buy a new one for $60.

Not now. In 10 minutes we found the exact replacement OEM remote for $25 including shipping. Not exactly a bargain, but much cheaper than buying new. The world economy just deflated by $35. Do you hear it hissing?

—